rTop-k: A Statistical Estimation Approach to Distributed SGD
نویسندگان
چکیده
منابع مشابه
Statistical inference using SGD
We present a novel method for frequentist statistical inference in M -estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling. An intuitive analysis using the OrnsteinUhlenbeck process suggests that such averages are asymptotically normal. From a prac...
متن کاملA statistical approach to classify Skype traffic
Abstract- Skype is one of the most powerful and high-quality chat tools that allows its users to use of many services such as: transferring audio, sending messages, video conferencing and audio for free. Skype traffic has a lot of Internet traffic. Hence, Internet service providers need to identify traffic to do the quality of service and network management. On the other hand, Skype developers ...
متن کاملRevisiting Distributed Synchronous SGD
Distributed training of deep learning models on large-scale training data is typically conducted with asynchronous stochastic optimization to maximize the rate of updates, at the cost of additional noise introduced from asynchrony. In contrast, the synchronous approach is often thought to be impractical due to idle time wasted on waiting for straggling workers. We revisit these conventional bel...
متن کاملAccurate RTOP Estimation from PFG-NMR Signal
The return to the origin probability (RTOP) for diffusing molecules is a valuable indicator of porous media microstructure [1-2]. For example, in isolated pores with nonrelaxing walls, the pore volume is related to the RTOP at long diffusion times. Similarly, in disordered media, the temporal scaling of the RTOP is necessary in the estimation of the fractal dimension of the medium [3]. However,...
متن کاملA Geometric Approach to Statistical Estimation
The role of Kerridge inaccuracy, Shannon entropy and Kullback-Leibler distance in statistical estimation is shown for both dicrete and continuous observations. The cases of data independence and regression-type dependence are considered in parallel. Pythagorean-like relations valid for probability distributions are presented and their importance for estimation under compressed data is indicated.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Journal on Selected Areas in Information Theory
سال: 2020
ISSN: 2641-8770
DOI: 10.1109/jsait.2020.3042094